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ABSTRACT 



Nearly all the states produce one or more accountability or 
indicator reports containing data reported at one or more levels of the 
education system as well as data from other sources. To inform discussion 
about what data are useful and appropriate to report, comparisons were made 
of state accountability and indicator reports. A list of 74 indicators was 
prepared that reflected indicators commonly used in 25 national and 
international indicator reports. Over 100 reports from 45 states, the 
District of Columbia, and the Department of Defense Dependents Schools were 
reviewed. Of the 47 states, 37 reported data on average subject area 
proficiencies by 1 or more grade levels, 33 states reported information about 
graduation rates, and 37 states reported dropout rates. Also frequently 
reported are financial data about per pupil expenditures and expenditures by 
function. This report does not vouch for the quality of any data reported. A 
major conclusion from this study is that indicators that appear to be the 
same on the surface, and even have the same name, can, in fact, be very 
different and can answer different questions. It is a good idea for a state 
or district that wants to establish indicators to review current practices in 
schools and districts before determining what indicators to collect and 
report. (Contains four tables.) (SLD) 
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WHAT DO WE KNOW ABOUT EDUCATION IN THE STATES? 
EDUCATION INDICATORS IN SEA REPORTS 

Barbara S. Clements 
Rolf K. Blank 



Currently, nearly all of the states produce one or more accountability or indicator reports. 
These reports contain data reported at one or more levels of the education system (state, district, 
or school), as well as data obtained from other sources, such as state departments of health and 
human resources and the U.S. Census Bureau. Many of these reports are patterned after the 
National Education Goals Panel’s annual Goals Report. Data reported include student 
enrollments, graduation rates, dropout rates, assessment scores, and other demographic and 
performance data. 

The purposes of these reports vary; some are a part of mandated reporting requirements 
by state legislatures and state boards of education, while others are to inform the public. Some 
reports contain data used to identify districts or schools needing assistance to improve student 
learning. In other instances, schools and districts received rewards for performance indicated in 
these reports. 

Chief state school officers and others working on the indicator reports have recognized 
the importance of choosing indicators “about the status or health of the educational system that 
can be readily, reliably, and repeatedly obtained.” 1 In the February 1997 Report to the State 
Board. Marc Hull, Education Commissioner in Vermont wrote, “Education isn’t a contest and 
data shouldn’t be used to set up competitions with winners and losers, rather it should be used to 
start conversations within communities.” (P. 1) 

In an effort to inform deliberations about what are useful and appropriate data to report 
about schools, districts, and states, comparisons were made of these state accountability/indicator 
reports. This paper summarizes the results of those comparisons. 

Methods 

State education agencies were asked to describe accountability and indicator reports 
produced for the 1994-95 and the 95-96 school years. In addition, they were asked to send 
copies of their accountability or indicator reports to the Council of Chief State School Officers 
(CCSSO). Information about these reports was summarized in 1995 and 1996 in documents 
titled State Education Accountability Reports and Indica tor Reports: Status of Reports Across 
the States produced by the CCSSO State Education Assessment Center. These summary reports 
contain information about the level of statistics reported, whether or not the reports are 
mandated, target audiences and other information. 



1 Committee for School Indicators, Massachusetts Association of School 
Superintendents. (1991) The Case for Local School Indicators. Author. P. i. 



Review of the accountability/indicator reports was done by staff of the National 
Elementary/Secondary Education Data and Information System (NESDIS) Project at CCSSO. 
NESDIS staff were working with the Core Data Task Force of the National Forum on Education 
Statistics to identify a set of essential data elements that should be included in all education 
information systems and that could be used to create indicators of education system functioning. 
As a part of this project, staff reviewed over one hundred reports received from state and local 
education agencies in 45 states, the District of Columbia and the Department of Defense 
Dependents Schools published since 1991. For each report, we noted the following things: 1) 
mandated or not, 2) cycle (e.g., annual), 3) level of statistics reported, 4) alignment with 
National Education Goals, 5) whether or not comparisons are made (e.g., with other states, the 
U.S. average), and 6) if data are reported from sources other than schools (e.g., health statistics, 
SAT scores). When reports for more than one year were available, we looked at changes in 
reporting over the years. For some of the more common indicators, we attempted to review 
definitions used in computing the indicators to see if they were comparable from state to state. 

To help organize the review of state-reported indicators, we used a listing of 74 
generically stated indicators which appeared in numerous national and state indicators reports. 
This list was compiled from a review of 25 national and international indicator reports. We 
defined indicators as the aggregate counts or computed rates that provide information about 
groups of students, teachers, schools, and districts. Indicators are used to answer questions about 
the status, functioning, and performance of the unit reported. For each state, we noted if they 
had the indicator stated exactly as the generic indicator, or if it was an approximate match. 

Since the goal was to identify commonly occurring indicators, having an exact match was not 
considered essential. The state summary table included in this document does not provide 
information for each state about whether or not the indicator is an exact match, that is, is defined 
the same. 

Results 

Status of State Education Accountability and Indicator Reports 

In 1995, 45 states and the District of Columbia indicated they produced at least one 
annual accountability or indicator report and 24 states indicated that they produced two or more 
reports. In 1996, the number of states producing accountability or indicator reports increased to 
49 (including the District of Columbia), of which 30 produced two or more reports. In 1996, 
only Minnesota and Wyoming indicated they do not produce reports; however, data usually 
included in these types of reports was found on their World Wide Web sites. Nebraska indicated 
that the local school districts were required to produce reports, not the state; but a written 
statistical report was actually published by Nebraska for the 1995-96 school year, perhaps after 
our survey was completed. Between 1995 and 1996, three states began producing 
accountability/indicator reports (Arkansas, Montana, and South Dakota); nine states increased 
the number of reports produced (Georgia, Kansas, Montana, Nevada, New Hampshire, Ohio, 
Utah, Vermont, and West Virginia); while one state, Oklahoma, appeared to have reduced the 
number of reports produced from six to three. 

In 1995, 41 states indicated that they reported district-level indicators, and 36 states 

2 



O 

ERIC 



t; 

kJ 



reported school-level indicators; whereas, in 1996, 45 states reported data at the district level, 
and 37 states reported data are the school level. In 1995, 36 states indicated that at least one of 
their reports were mandated, while in 1996, 38 states produced mandated reports 

States varied in how many printed reports were produced. Some states produced large 
number of reports for distribution to districts, principals, teachers, legislators, and the public; 
other states produced only enough to distribute to schools and districts. Several states indicated 
that they afe now providing the data through the Internet (Indiana, Maine, Michigan, and 
Pennsylvania). A review of state home pages on the Internet indicated that 30 of the 49 states 
with web sites have at least some data listed on their home pages that might typically be included 
in an accountability/indicator report. (The South Dakota Department of Education and Cultural 
Affairs’ web site could not be located.) Three additional states had sections under development 
to list school, district, or state data. 

Thus it appears that the demand or perceived need for published data about schools and 
school districts seems to be growing, and states are finding it useful to put these data on the 
Internet as well as in published documents. 

Contents of State Indicators Reports 

The review of state indicator reports was focused around the 74 indicators occurring 
most frequently in national and international indicator reports. The following table, titled 
Education Indicators, contains the listing of the generic indicators obtained from the review of 
national and international reports. For each generic name of an indicator, an attempt has been 
made to determine if state education agencies have reported data on the indicator in one or more 
of their indicator reports between 1991-1995. 

In this table it can be seen that 37 of the 47 states reported data on average subject area 
proficiencies by one or more grade levels (such as NAEP or state assessment scores). Thirty- 
three states reported data about graduation rates, and 37 states reported drop-out rates. Also 
frequently reported are financial data about per pupil expenditures and expenditures by function. 
On the other hand, no states reported data about the percent of children receiving routine health 
care or about the number of vandalism occurrences in schools. 

States obtain data from many sources, but primarily from schools and school districts. 
Some states report data obtained from the state’s office of vital statistics, the Census Bureau and 
other agencies or organizations. One cannot tell from this table where the data came from, 
merely that the state included data on the indicator in a published report and at some level (state, 
school district or school). 

The number of exact matches added to the number of approximate matches does not 
always equal the total number of states reporting. This is because some states provided more 
than one indicator report. Therefore, a state may have an exact match of an indicator on one 
report and an approximate match on the same indicator in another report. 

The second table, titled Background and Process Indicators, contains a list of the 17 
most frequently occurring background and process indicators in state education agency reports. 
For each state whose recent indicator reports were reviewed, the word “yes” indicates that the 
indicator or a similar indicator occurred in one or more of that state’s indicator reports that we 
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reviewed. Similarly, the third table, titled Outcome Indicators, contains a list of the 12 most 
frequently occurring outcome indicators in state education agency indicator reports. Once again, 
an indication is provided if the indicator or a variation of it appears in one or more of the state’s 
indicator reports. 

A comparison was also done of the formulas/definitions used to compute graduation rates 
included in the state reports or described by states in documentation sent to CCSSO. It is 
inappropriate to compare data across states if the formulas or definitions used to collect data are 
different. In addition, it is useful to review how states are computing data if a state or district is 
considering the most appropriate way to report data. The following table, titled Comparisons of 
Graduation Rate Formulas, shows the three main ways states compute graduation rates, if they 
compute them. Of the 35 states for whom we had information, 14 states compute a cohort 
graduation rate, six states compute a leaver graduation rate, and seven states compute a twelfth 
grade event rate. Fourteen states indicated they do not compute graduation rates. Reviewing 
this information is useful in considering whether to promote a single method of computing 
graduation rates so that comparisons can be made. This information can serve as a warning to 
any group seeking to develop cross-state graduation rate comparisons using existing data, 
because comparisons of the rates would be spurious and unfair. 

Conclusions 

The information included in this paper was originally produced for two reasons. The 
status of state accountability and indicator reports information was collected and published so 
that states, districts, or others interested in indicators could learn more about what states are 
doing and find out how to get copies of the documents. The lists of indicators and whether or 
not states report them were produced to summarize what indicator data are currently collected 
and reported by states, and to inform decision-making about a useful set of indicators to report 
on public education. Together, these two sets of information provide a description of what data 
are being collected by state education agencies for the purpose of informing the public or 
addressing reporting mandates. 

We cannot vouch for the quality or comparability of the data collected from schools, 
districts, and other sources within or across state lines. However, quality and comparability are 
essential if schools and districts (or states) are to be compared so that rewards or punishments 
can be administered or if consistency and improvements over time are to be measured. In 
addition, it is important to ensure that legislators and the public are not deceived because the 
data are not comparable. We have learned through our work that indicators that appear to be the 
same on the surface, that is they are named the same, can actually be very distinct, and answer 
very different questions. Thus, it is a good idea for a state or district wanting to establish 
indicators to review what are current practices in schools and districts, and even other states, 
before determining what indicators to collect and report, how to define the indicators, and when 
to collect the data. 
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74db# 


Domain 


Generic Name 


# Exact 
Matches 


# Approximate! 
Matches 


| Total# 
States 
(Reporting 


0020 


Background 1 


% of children with iimited English proficiency 


11 


6 


13 


0030 


Background 


% of children with a home language other than 
English 


4 


2 


6 


0080 


Background 

i 


% of children by family type 


0 


5 


5 


0081 


Background 


% of babies with low birth weight 


5 


4 


7 


0085 


Background 


% of children receiving recommended immunizations 


1 


5 


6 


0086 


Background 


% of children receiving routine health care 


0 


0 


0 


0090 


Background 


Highest level of education completed by 
parents/guardians 


1 


5 


6 


0100 


Background 


% of families with children receiving public 
assistance 


0 


23 


23 


0110 


Background 


Poverty status of the community 


1 


12 


12 


0120 


Background 


% of children living in severely distressed 
neighborhoods 


0 


1 


1 


0130 


Background 


Average family income (scaled) 


5 


13 


15 


0140 


Background 


% of mothers receiving prenatal care 


1 


5 


6 


0145 


Background 


% of students enrolled having preschool experiences 


0 


6 


6 


0405 


Background 


% of students enrolled by subject area by course title 
by grade level 


8 


15 


18 


0406 


Background 


Mobility rate for students 


6 


11 


14 


0548 


Background 


% of students by race/ethnicity by gender 


8 


27 


31 


0625 


Background 


% of students receiving special sen/ices (English as 
a Second Language, Special Education, Title 1) 


9 


25 


29 


0626 


Background 


% of students employed by number of hours/week by 
age 


0 


1 


1 


0170 


Outcomes 


% of students taking AP courses by subject area by 
grade level 


7 


4 


10 


0190 


Outcomes 


% of students scoring 3+ on AP tests 


9 


7 


14 


0200 


Outcomes 


Average subject area proficiencies by grade level (e. 
g., NAEP) 


27 


29 


38 


0205 


Outcomes 


% of students meeting state and local achievement 
standards by school, income, grade level, R/E, etc 


27 


11 


29 


0206 


Outcomes 


Average scores on college entrance exams 


35 


9 


37 


0229 


Outcomes 


% students taking specified core curr. (e.g., 4 Math, 3 
Sci, 4 English 


2 


8 


9 


0230 


Outcomes 


% of students graduating from high school 


32 


11 


34 


0240 

. 

I c 


Outcomes 


% of students graduating with a certificate of 
completion 


10 


5 


14 
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74db# 


Domain 


Generic Name 


# Exact 
Matches 


# Approximate 
Matches 


Total# 

States 

Reporting 


0250 


Outcomes 


% of students completing high school equivalency 
requirements 


9 


9 


\7 


0255 


Outcomes 


% of students who dropped out 


39 


9 


39 


0280 


Outcomes 


% HS grads enrolled in postsec ed by institution type 
(2-/ 4-yr/ VocEd) immediately after graduation 


8 


11 


14 


0300 


Outcomes 


Employment rate for recent high school graduates 
who did not go on to college 


1 


8 


9 


0305 


Outcomes 


% of students taking college entrance exams 


28 


5 


30 


0307 


Outcomes 


Absenteeism rate for students 


1 


6 


7 


0545 


Outcomes 


Student attendance rate by type of program 


28 


9 


30 


0546 


Outcomes 


% of students enrolled by type of program 


16 


20 


27 


0547 


Outcomes 


Average number of hours spent in extracurricular 
activities 


0 


4 


4 


0306 


Process 


# of students obtaining in-school coop worksite 
placements 


0 


0 


0 


0400 


Process 


% of students suspended or expelled as a 
disciplinary action 


2 


5 


7 


0410 


Process 


% of students receiving selected instructional 
practices by subject 


0 


6 


6 


0430 


Process 


# of articulation agreements that meet TechPrep 
standards 


0 


0 


0 


0450 


Process 


Average class size by subject area 


11 


2 


12 


0460 


Process 


# of courses taught by multiple teachers/staff 


0 


0 


0 


0465 


Process 


# of courses taught jointly by academic and 
occupational education teachers 


0 


1 


1 


0480 


Process 


Adult/child ratio of preschool, kindergarten, and 
elementary programs 


1 


0 


1 


0490 


Process 


Teacher/student ratio by program type or by subject 
area or by grade level 


27 


8 


27 


0500 


Process 


Average # of hours/week spent in different subject 
areas 


0 


7 


7 


0520 


Process 


Ratio of time spent in core subjects to time in school 


1 


2 


2 


0530 


Process 


# of school days per school year 


6 


1 


7 


0540 


Process 


Extent of parental involvement in preschool programs 
and schools 


0 


7 


7 


0560 


Process 


Graduation requirements (courses and tests by 
subject area, etc.) 


7 


3 


8 


0590 


Process 


Student suspension/expulsion/attendance policies 


0 


0 


0 


0600 


Process 


# of criminal offenses reported by type 


0 


7 


7 


0610 

1C 


Process 


# of vandalism occurrences 


0 


0 


0 
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74db# 


Domain 


Generic Name 


# Exact 
Matches 


# Approximate 
Matches 


Total # 

States 

Reporting 


0620 


Process 


Student-counselor ratio 


3 


1 


4 


0630 


Process 


% of students using substances at school by grade 
level 


0 


8 


8 


0640 


Process 


Characteristics of teaching assignments by school 
type 


0 


27 


27 


0660 


Process 


Extent of teacher participation in professional 
development activities by content and type 


0 


9 


9 


0670 


Process 


Extent of preservice training by instructional level 


0 


0 


0 


0645 


Resources 


% of teachers assigned to teach within field of 
certification by subject area 


4 


2 


6 


0646 


Resources 


% of students with a non-certified teacher by subject 
area 


0 


3 


3 


0720 


Resources 


Per pupil expenditure 


29 


17 


35 


0725 


Resources 


Expenditure by function 


23 


25 


34 


0750 


Resources 


Average salary by staff type by level of experience 


7 


29 


30 


0760 


Resources 


% of staff by race/ethnicity and/or by sex 


5 


20 


22 


0770 


Resources 


% of staff who can speak a non-English language by 
language 


0 


0 


0 


0775 


Resources 


Revenue by source 


21 


24 


31 


0780 


Resources 


Availability of support services for students 


1 


4 


5 


0790 


Resources 


Extent of school and community coordination by 
service type 


0 


1 


1 


0870 


Resources 


Rate of staff turnover by instructional level 


10 


4 


11 


0880 


Resources 


# of teaching positions filled with non-certified 
personnel 


2 


1 


2 


0890 


Resources 


Teacher education by subj area by grade level 


17 


19 


27 


0896 


Resources 


Extent of usage of computers during instruction 


0 


6 


6 


0897 


Resources 


% of students with access to computers in school by 
grade level 


1 


5 


5 


0898 


Resources 


# of students per academic computer 


2 


2 


3 


0899 


Resources 


% of teachers with access to computers in school 


0 


1 


1 
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